1. Field of the Invention
The present invention relates to an image recognition device, an image recognition method, and an image recognition program.
2. Background Art
Recently, an adaptive cruise control (ACC) system, a forward collision warning (FCW) system, a pedestrian collision warning system, and the like have been developed as a driving support system or a preventive safety system of a vehicle. The distribution of low-cost systems using an on-board camera is expected.
Pattern recognition has been often used for recognition of an object using an on-board camera (for example, see Japanese Unexamined Patent Application, First Publication No. 2007-310805 (Patent Document 1)).
An object recognition algorithm using the pattern recognition is constructed by combining classifiers of AdaBoost, SVM (Support Vector Machine), or the like with feature values of Haar-like, HOG (Histograms of Oriented Gradients), or the like.
In the pattern recognition, in order to extract a target object (an image of the target object) from a captured image, an image region (window) from which an object is recognized is set to various sizes and the object recognition algorithm is performed for each window.
As another approach, a method of estimating the three-dimensional structure of a traveling environment based on an on-board stereoscopic camera or monocular camera and detecting an object is also considered.
For example, when an on-board monocular camera is used, a technique (which is referred to as a 3D analysis technique in this specification) using acquired time-series images is known (for example, see Japanese Unexamined Patent Application, First Publication No. 2011-146017 (Patent Document 2)).
In the 3D analysis technique, feature points, optical flows, and the like are calculated from images (for example, an image at time t and an image at time t−1) having a time difference, the motion of a traveling vehicle is estimated, and then a target object, an obstacle, or the like is detected. Here, the pattern recognition is more advantageous in performance for recognizing a target object.
A pattern recognition process which is performed by an object recognition unit (for example, a processing unit corresponding to an object recognition unit 13 shown in FIG. 1) according to the background art will be described below with reference to FIG. 18.
FIG. 18 is a flowchart illustrating an example of a process flow which is performed by an object recognition unit according to the background art.
In this example, a recognition algorithm is constructed by HOG feature values and real AdaBoost classifiers.
First, the object recognition unit performs a process of calculating gradient directions and gradient magnitudes in all acquired intensity images and integrating the gradient magnitude of each gradient direction, and calculates an integral gradient image (integral histogram) which is a result of the integration (step S1001). Accordingly, the integral gradient images corresponding to the number of gradient directions are calculated.
Then, the object recognition unit extracts a region of the integral gradient image of each gradient direction through the raster scanning while causing a coordinate region (window) with a predetermined scale (size) to slide (step S1002).
When an integral gradient image is used, images to be raster-scanned are required by the number of gradient directions and thus the processing load thereof is heavier than when an integral image, for example, using a Haar-like feature value is raster-scanned.
In this manner, when the Haar-like feature value is used instead of the HOG feature value, the process can be completed by preparing a single integral image using a general integral intensity image instead of the integral gradient image and it is thus possible to save the processes. However, for example, only features such as an intensity difference between specific regions can be extracted and thus features for each gradient direction cannot be known.
Subsequently, the object recognition unit calculates the HOG feature value (vector) of the extracted coordinate region (window) (step S1003). Accordingly, a gradient histogram is created by cells.
Then, the object recognition unit performs classification with a real AdaBoost classifier by the use of the calculated HOG feature value (vector) and recognizes an object (an image of the object) which is previously set as a target (step S1004).
Here, the object recognition unit determines whether a series of raster scans has completed (step S1005).
Then, the object recognition unit ends the process flow when it is determined that a series of raster scans has completed.
On the other hand, when it is determined that a series of raster scans has not completed, the object recognition unit causes the window to shift (to slide) over a raster scan region and performs the process of step S1002.
In this manner, the object recognition unit causes the window to sequentially slide over the raster scan region and carries out repeated performance of the processes of step S1002 to step S1004, until a series of raster scans has completed.
In the series of raster scans, for example, the processes of causing a window with a fixed scale (size) to sequentially slide over an image region, carrying out repeated performance of the above-mentioned process, then changing the scale or the moving step (scanning step) of the window, causing the window to sequentially slide, and carrying out repeated performance of the above-mentioned process are performed a predetermined number of times. Accordingly, conversion into a feature vector is carried out.
The integral gradient image may not be used to create the gradient histogram, but the method using the integral gradient image can reduce the processing time, which is effective.
A rough example of the process flow of from the acquisition of an image to the raster scan in the object recognition process flow shown in FIG. 18 will be described below.
FIG. 2 is a diagram illustrating a rough example of the process flow from the acquisition of an image to the raster scan.
Data of an image captured by a camera (for example, corresponding to a camera 11 shown in FIG. 1) according to the background art is acquired as data of an input image 101 by an image acquiring unit according to the background art (for example, a processing unit corresponding to an image acquiring unit 12 shown in FIG. 1).
For example, a road and a preceding vehicle appear in this image.
An object recognition unit (for example, a processing unit corresponding to an object recognition unit 13 shown in FIG. 1) according to the background art calculates a gradient magnitude of each of multiple gradient directions (eight directions of direction 1 to direction 8 in this example) as shown in a gradient magnitude image 102.
Then, the object recognition unit creates an integral image of the gradient magnitude of each gradient direction as an integral gradient image 103 (the process of step S1001 shown in FIG. 18).
The object recognition unit scans a raster scan region 104 with a window (the process of step S1002 shown in FIG. 18).
In the raster scan, since it is necessary to raster-scan the integral gradient images of all the directions, the processing load increases.
The changing of a scale and the changing of a moving step (step width) of the raster scan in the process of step S1002 shown in FIG. 18 will be described below.
In the raster scan, the scanning operation is performed, for example, while changing a scale SC and moving steps dx and dy.
Part (A) and Part (B) of FIG. 3 are diagrams illustrating examples of the scale and the moving step of the raster scan.
In this example, it is assumed that there are four or more patterns of combinations of the scale and the moving step of the raster scan.
Part (A) of FIG. 3 is a diagram illustrating an example of the scale SC1 and the moving steps dx1 and dy1 of the raster scan in a first pattern.
In this example, in a raster scan region 201, a square window 202 having the length of the scale SC1 in the x axis direction as the transverse (horizontal) direction and the y axis direction as the longitudinal (vertical) direction is set to a predetermined initial position. This window 202 is made to slide in the x axis direction by the moving step dx1 and to slide in the y axis direction by the moving step dy1.
Part (B) of FIG. 3 is a diagram illustrating an example of a scale SC4 and moving steps dx4 and dy4 of the raster scan in a fourth pattern.
In this example, in a raster scan region 211, a square window 212 having the length of scale SC4 in the x axis direction as the transverse (horizontal) direction and the y axis direction as the longitudinal (vertical) direction is set to a predetermined initial position. This window 212 is made to slide in the x axis direction by the moving step dx4 and to slide in the y axis direction by the moving step dy4.
In this example, the values of the scale SC1 and the moving steps dx1 and dy1 of the raster scan in the first pattern are different from the values of the scale SC4 and the moving steps dx4 and dy4 of the raster scan in the fourth pattern.
In this example, for example, the scales SC of the raster scans in the patterns are different and the moving steps dx and dy having a magnitude proportional to the scale SC are used. In this example, the moving step dx in the x axis direction and the moving step dy in the y axis direction in each pattern are set to the same value.
In another configuration example, a window having a shape (for example, a rectangle other than a square) other than a square may be used. In this specification, a rectangle includes a square as well as a rectangle.
In another configuration example, the moving step dx in the x axis direction and the moving step dy in the y axis direction in a certain pattern may be set to different values.